In video and imaging systems, color generally is represented as vector coordinates in a three dimensional “color space.” Common examples include the well-known classes of RGB and YUV color spaces. The RGB color spaces specify pixel values using coordinates that represent intensities of red, green and blue light, respectively. The YUV color spaces specify pixel values using coordinates that represent a luminance or chrominance value.
Due to the nonlinear response of cathode ray tube (CRT) monitors (also known as the monitor's gamma), data is often specified as intensity values relative to the inverse of the gamma. For example, in YUV color spaces, actual values stored relative to the inverse gamma (typically denoted using an apostrophe) are referred to as the “luma” value (Y′) and “chroma” values (U′ and V′). RGB data stored relative to the inverse gamma is typically denoted using the notation R′G′B′. This form of data is also known as ‘nonlinear’ YUV or RGB data.
When YUV data is stored in a digital or analog form, the ranges for each of the components are typically distinguished using a variation of the YUV notation. For example, the notation Y′Pb′Pr′ refers to nonlinear YUV data where the Y component is in the range of [0 . . . 1] (0 being black, 1 being white) and the chroma components range from [−0.5 . . . 0.5]. As another example, data stored in an 8-bit or 10-bit fixed point approximation is denoted using the notation Y′Cb′Cr′.
For video, pixel data is typically represented using the YUV color spaces, which can be derived from RGB information. Conversion between the RGB and YUV color spaces typically involves a simple transform. For example, the Y′Pb′Pr′ color data is a variation of the R′G′B′ color space where the Y′ component is proportional to the perceptual brightness (Y′=0.299R′+0.587G′+0.114B′), and the Pb′ and Pr′ components are defined as color difference from the brightness (e.g., Pb′=B′−Y′; Pr′=R′−Y′).
When Y′Pb′Pr′ is stored in an 8-bit approximation, the Y′ range [0 . . . 1] is mapped to the range 16 . . . 235. The chroma components Pb′ and Pr′ are mapped to their corresponding Cb′ and Cr′ components by mapping the range of [−0.5 . . . 0.5] to an interval of width 224 centered about 128 (i.e. 16 to 240). For example, an 8-bit representation is computed as:Y′=16+Y′*219Cb′=128+Pb′*224Cr′=128+Pr′*224
Higher bit precisions are computed by simply scaling up the 8-bit values. For example, an n-bit representation is computed as:Y′=(16+Y′*219)*2n-8 Cb′=(128+Pb′*224)*2n-8 Cr′=(128+Pr′*224)*2n-8 
Pixel data can be stored in a packed format or planar format. In a packed format, the components corresponding to a given pixel are stored as a cluster or group in a single array in memory, and data for all components can be obtained in a single read. Packed formats can be convenient when performing operations on an entire pixel, but can be inefficient if an operation on a single channel (e.g., an operation on the R channel for an RGB pixel) is desired.
For example, in FIG. 1 an RGB pixel in an 8-bit per channel representation is stored in packed format as a little-endian DWORD 100. The data for the RGB pixel comprises 24 bits of color information in adjacent bytes in memory (one byte each for the R, G, and B channels), along with an 8-bit alpha (or transparency) value. Because DWORD 100 is little-endian, Byte 0 (which contains the alpha value) appears on the far right as the least significant byte. As shown in FIG. 1, in an 8-bit representation a pixel with three full-resolution channels of color information will have 24 bits of color data. For memory alignment and access efficiency reasons, pixel data is often stored on a 16- or 32-bit boundary. Therefore, a pixel with 24 bits of color data is typically stored along with an 8-bit alpha value, so that the pixel data for each pixel occupies 32 bits.
In a planar format, information from different channels is stored in separate planes, rather than being grouped into a single array. For example, in FIG. 2 data for an RGB image 200 stored in an R plane 210, a G plane 220, and a B plane 230. In an 8-bit representation, each pixel would include 8 bits of information from each plane. Planar formats can be inefficient when performing operations on an entire pixel because getting data for an entire pixel requires three reads—one for each plane. However, planar formats can be convenient in other situations, such as when performing an operation on a single channel (e.g., filtering out all of the red in an RGB image).
Because human perception is not as sensitive to color changes as it is to brightness changes, chroma components can be sampled at a lower spatial resolution than the luma components with little perceived loss in image quality. For example, in a “4:2:2” sub-sampling format, a pair of chroma samples (e.g., a pair of Cr′ and Cb′ samples) is “shared” between two luma samples. In other words, a 4:2:2 image has one pair of chroma samples for every two luma samples—the chroma samples are sub-sampled by a factor of two in the horizontal direction.
Table 1 describes several common sub-sampling formats.
TABLE 1Sub-sampling formatsSub-samplingformatDefinition4:4:4For every luma sample, there is a pair ofcorresponding chroma samples (e.g., a Cb' andCr' sample).4:2:2For every two horizontal luma samples, there isa pair of corresponding chroma samples.(Horizontally, there is half as much chromainformation as luma information.)4:1:1For every four horizontal luma samples, there isa pair of corresponding chroma samples.(Horizontally, there is a quarter as muchchroma information as luma information.)4:2:0For each 2 × 2 square of luma samples, there is acorresponding pair of chroma samples.(Chroma information is sampled both verticallyand horizontally at half the resolution of theluma samples.)
For more information on color spaces and sub-sampling formats, see, e.g., Keith Jack, Video Demystified: A Handbook for the Digital Engineer, LLH Technology Publishing (3d ed. 2001).
Whatever the benefits of previous techniques, they do not have the advantages of the following tools and techniques.